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ABSTRACT 

The primary purpose of this paper is to review how BEST in CLASS, an 
evidence-based program targeting children at risk for emotional and beha- 
vior disorders, interacts with a diverse range of social ecological factors to 
promote successful implementation. Recent findings from two BEST in 
CLASS studies (BEST in CLASS-Prekindergarten and BEST in CLASS- 
Elementary) are reviewed in relation to barriers and facilitators of high 
quality implementation. The importance of taking into account factors 
associated with teacher delivery of interventions in classrooms, particularly 
for children with and at-risk for emotional and behavioral disorders, when 
designing and evaluating interventions is discussed. Implications for future 
research and practice are provided. 


Classrooms are embedded in a social ecology in which aspects of the individual child, teacher, and 
classroom all influence one another (Bronfenbrenner, 1992). Learning about the extent to which 
these factors influence implementation quality will enhance our ability to design and refine inter- 
ventions that more effectively match with contextual and individual level factors that may support 
high quality implementation (Durlak, 2010). This is particularly important when designing and 
assessing interventions that target children at risk for emotional and behavioral disorders (EBD). 

Approximately 11.5 - 30% of children in the U.S. display symptoms of chronic problem behavior 
(e.g., disruptive behavior, inappropriate emotional responses, defiance) that place them at risk for 
developing EBD (Forness, Kim, & Walker, 2012; Qi & Kaiser, 2003; Ringeisen et al., 2017). Children 
who are at risk for or have EBD have difficulties with adjustment at school and these difficulties are 
likely to persist throughout child and adulthood (Phillips & Shonkoff, 2000). For instance, the early 
onset of behavior problems in young children predicts problems in adolescence including juvenile 
delinquency and school dropout (Webster-Stratton, 2000). Moreover, EBDs are predictive of a 57% 
likelihood of being arrested within 2 years of leaving high school, employment instability, and risk of 
requiring mental health treatment (Newman, Wagner, Cameto, & Knokey, 2009). Considering these 
adverse outcomes, it is no surprise identifying factors that promote or hinder psychosocial adjust- 
ment for children with and at risk for EBD is a national health priority. 

High quality early childhood and elementary classroom environments can build resiliency that 
can serve as a protective mechanism to promote psychosocial adjustment for children with and at 
risk for EBDs (Curby, Rudasill, Edwards, & Pérez-Edgar, 2011; Mokrova et al., 2015; Rudasill et al., 
2016). Children spend a large part of their day in early childhood or elementary school classrooms, 
and their experiences in the classroom have the potential to strengthen children’s resilience from 
a myriad of risk factors (Pianta & Walsh, 1996). Sameroff’s transactional model (Sameroff, 2009) 
helps us understand the classroom as a context in which teachers and children reciprocally influence 
one another through interactions, and these interactions influence subsequent behaviors and future 
interactions. This makes early childhood and elementary school classrooms an ideal context to 
support children with and at risk for EBD, and also highlights the salience of teachers’ influence on 
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children’s development and school performance (Pianta, 1999; Wentzel, 2002). For instance, teacher- 
child relationships characterized by warmth and closeness are related to improvements in children’s 
self-regulation skills at the end of the school year (Cadima, Verschueren, Leal, & Guedes, 2016). 
Additionally, high quality classroom interactions, characterized by high levels of emotional support 
and classroom organization, are associated with children’s skills in inhibitory control, phonological 
awareness, and print knowledge (Hatfield, Burchinal, Pianta, & Sideris, 2016). 

Given these findings, the primary purpose of this paper is to review how BEST in CLASS, an 
evidence-based program targeting children at risk for emotional and behavior disorders (EBD), 
interacts with a diverse range of social ecological factors to promote successful implementation in 
classrooms. 


BEST in CLASS 


BEST in CLASS is framed by the transactional model (Sameroff, 2009) and is a teacher delivered 
intervention that capitalizes on the potential of supportive classroom environments to improve 
children’s psychosocial adjustment through teacher-child interactions. BEST in CLASS promotes 
teachers’ increased use of effective instructional practices through professional development, includ- 
ing individualized practice-based coaching, which promote the key mechanisms of change: positive 
teacher-child interactions and improved teacher-child relationships. BEST in CLASS is a Tier-2 
(secondary) intervention as teachers are trained and coached to increase their frequency of use and 
quality of delivery of BEST in CLASS instructional practices with children systematically identified as 
at risk for EBD. BEST in CLASS has been shown to be effective at improving child (Conroy et al., 
2015; Sutherland, Conroy, Algina et al., 2018) and teacher outcomes and has been named an effective 
program by the National Institute of Justice (see https://www.crimesolutions.gov). Currently, there 
are two versions of BEST in CLASS: BEST in CLASS-Prekindergarten (BEST in CLASS-PK) and 
BEST in CLASS Elementary (BEST in CLASS-E). 

Both BEST in CLASS-PK and BEST in CLASS-E are Tier-2 prevention programs designed to reduce 
problem behavior of children with and at risk for EBD who are not responsive to Tier-1 (i-e., universal) 
supports. Children are systematically screened and identified as being at an increased risk for EBD. For 
screening, within each classroom, teachers identified one to three children to participate based on 
externalizing problem behavior that interfered with participation in the classroom and risk for EBD 
using the Early Screening Project (ESP; see Feil, Severson, & Walker, 1998) for BEST in CLASS-PK or 
the Systematic Screening for Behavior Disorders (SSBD; Walker, Severson, & Feil, 2014) in BEST in 
CLASS-E. BEST in CLASS is designed to be delivered intentionally by teachers to these focal students 
(who have met screening criteria) during typically occurring instructional activities in the classroom. 
Both BEST in CLASS-PK and BEST in CLASS-E include a one-day interactive teacher training on the 
key instructional practices. At this training teachers are given a teacher manual that summarizes 
training content on the practices and serves as a reference guide for teachers throughout the program. 
BEST in CLASS-PK is comprised of six key practices: (a) Rules, (b) Behavior-Specific Praise; (c) 
Precorrection; (d) Opportunities to Respond; (e) Corrective Feedback; and (f) Instructive Feedback. 
A final module, Linking and Mastery, helps teachers efficiently and effectively link practices together 
to maximize the effect of the practices. Each of the six practices includes a framework and strategies to 
facilitate positive communication and engagement with children’s family members. Following the one- 
day training, teachers begin 14 weeks of practice-based coaching (see Conroy et al., 2015; Sutherland, 
Conroy, Vo, & Ladwig, 2015 for details of the training and coaching procedures). 

BEST in CLASS-PK has demonstrated efficacy at improving desirable teacher behaviors, reducing 
focal children’s problem behaviors, and increasing focal children’s desirable social and behavioral 
outcomes. Results from an efficacy trial that employed a multi-site cluster randomized design 
(Spybrook & Raudenbush, 2009) conducted with 186 teachers and 469 children randomly assigned 
to BEST in CLASS - PK or business as usual indicated (a) significant increases in teachers’ use of 
instructional practices, self-efficacy, and overall classroom quality (Conroy, Sutherland, Algina et al., 
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2019), and (b) reductions in problem behavior and improvements in teacher-child interactions and 
relationships (Conroy, Sutherland, Algina, Werch, & Ladwig, 2018; Sutherland, Conroy, Algina et al., 
2018). Improvements in observed child behavior noted in a previous study (Conroy, Sutherland, Vo, 
Carr, & Ogston, 2014) were supported by teachers’ reports of decreased problem behavior 
(ES = — .42) and externalizing problems (ES = — .42) associated with BEST in CLASS-PK. In 
addition, positive effects on all three dimensions of the CLASS (Pianta, La Paro, & Hamre, 2008) 
were noted (ESs of .47 for Emotional Support, .58 for Classroom Organization, and .65 for 
Instructional Support), highlighting the benefits of the program for all children in the classroom. 

BEST in CLASS-E was adapted from the initial framework of BEST in CLASS-PK. While results 
from the initial investigations of BEST in CLASS-PK confirm the efficacy of the intervention, the 
educational needs of early elementary-aged students and the professional development needs of their 
teachers are different from those in preschool (Pianta & Rimm-Kaufman, 2006; Rathus, 2008). BEST 
in CLASS-E involved several key changes from BEST in CLASS-PK. These adaptations are detailed 
below in reference to barriers and facilitators of implementation. However, most notably BEST in 
CLASS-E includes some different practices (described below as practice elements; Sutherland, 
Conroy, McLeod, Kunemund, et al., 2019) deemed more developmentally appropriate for elemen- 
tary-aged students: (a) Supportive Relationships, (b) Rules, (c) Precorrection, (d) Opportunities to 
Respond, (e) Behavior-Specific Praise, and (f) Home-School Partnership. 

In a small pilot study, BEST in CLASS-E demonstrated promise at improving teacher-child 
interactions and increasing focal students’ desirable behavioral outcomes (Sutherland Conroy, 
McLeod, Granger, et al., 2020). The pilot study was conducted with 45 kindergarten to 3"? grade 
students (identified at risk for the development of EBD) and their 26 teachers from three elementary 
schools located in an urban school district. Teachers and students were randomly assigned to BEST 
in CLASS - E or business as usual. Results indicated BEST in CLASS-E decreased teacher reported 
student problem behavior (ES = — .32) and increased teacher-student closeness (ES = .55). In 
addition, teachers reported reductions in emotional exhaustion (ES = — .35; Conroy et al., 2019). 
Results suggest the promise of BEST in CLASS-E as a Tier-2 intervention delivered by teachers in 
elementary classrooms. 


Implementation of BEST in CLASS 


Implementation is conceptualized as the way a program is put into practice and delivered to 
participants (McLeod, Kunemund, Nemer & Lyon, 2019). The assessment of implementation 
integrity can capture the dissonance between the theory or design of a program and what the 
program looks like when it is being implemented (Sutherland, McLeod, Conroy, & Cox, 2013). The 
assessment of implementation has an important influence on interpreting associated program out- 
comes. For instance, poorly implemented programs are often found to have little to no positive 
influence on outcomes (Durlak, 2010; Durlak & DuPre, 2008). Additionally, when the same program 
has a different influence on outcomes in different settings (e.g., across teachers, schools), the 
assessment of implementation can signal reasons for varying levels of program success across 
settings. Therefore, measuring and evaluating implementation quality is an essential indicator 
needed for the future development, evaluation, and sustainability of evidence-based interventions 
(Durlak, 2010, 2016). 

Treatment integrity is a measure of the degree to which core elements of an evidence-based 
intervention were implemented as intended (McLeod, Southam-Gerow, Tully, Rodriguez, & Smith, 
2013; Sutherland et al., 2013). While there are different conceptual models of treatment integrity 
measurement (e.g., stages of implementation; Fixsen, Blase, Naoom, & Wallace, 2005; observational 
assessments; Mowbray, Holter, Teague, & Bybee, 2003), we focus on four common dimensions of 
treatment integrity proposed in a conceptual model of program implementation for young children 
at risk for EBD (Figure 1): treatment adherence, competence of delivery, treatment differentiation, 
and relational factors (Sutherland et al., 2013). Treatment adherence is the extent to which an 
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Structure Teacher Implementation of EBP Outcomes 


Treatment Adherence 


Policy 


Professional Development 
Administrative Support Prosocial behavior 
Treatment Differentiation 
Teacher Characteristics 


Academic skills 


Child Characteristics Reduced EBD symptoms 
Relational Factors 


Figure 1. Conceptual model of treatment implementation. 


EBP = Evidence-based practice. EBD = Emotional behavioral disorder 


evidence-based practice is delivered as designed (i., the delivery of prescribed interventions). 
Treatment competence is the level of skill and degree or responsiveness demonstrated by 
a participant (e.g., a teacher) when delivering the prescribed intervention. Treatment differentiation 
is the extent to which the implementation of the intervention differed along appropriate lines 
defined by the protocol, while relational factors involve aspects of the intervention receipt (e.g., 
teacher-child closeness, child engagement). Measuring each of these integrity components allows 
researchers to answer important questions about the program validity needed to support program 
evaluation (Carroll et al., 2000). For instance, assessing adherence and competence allows research- 
ers to establish if a program was delivered as designed and if there was variation in delivery across 
teachers, classrooms, or schools. 

To date, evaluations of most evidence-based programs have lacked adequate measurement for 
evaluating and monitoring program implementation and teacher training (Sanetti, Dobey, & Gritter, 
2012; Sutherland et al., 2013). In response to this need, in BEST in CLASS we developed observa- 
tional treatment integrity measures, one for BEST in CLASS-PK and another for BEST in CLASS-E, 
in order to assess teacher implementation of BEST in CLASS practices. The BEST in CLASS 
Adherence and Competence Scale (BiCACS; Sutherland, McLeod, Conroy, Abrams, & Smith, 
2014) was developed as a treatment integrity measure to assess the adherence and competence of 
delivery of BEST in CLASS-PK practices by teachers. Both subscales (Adherence and Competence) 
assess the six teacher-delivered components core to the BEST in CLASS-PK model (Rules, 
Precorrection, Opportunities to Respond, Behavior-Specific Praise, Corrective Feedback, 
Instructive Feedback) using a seven-point Likert-type scale ranging from “1 = Not At All” to 
“7 = Very Extensive” for the Adherence Scale and “1 = Very Poor” to “7 = Excellent” on the 
Competence Scale. Scores for both adherence and competence were created by calculating the mean 
score for all observations of the BiCACS items then averaging together the items on each subscale 
with higher scores indicating higher levels of adherence and competence. 

The Treatment Integrity Instrument for Elementary School Classrooms (TIES) was developed for 
use in BEST in CLASS-E. This observational measure aligns closely with the previously developed 
BiCACS with three key differences. First, it was adapted for the practices included in BEST in 
CLASS-E (i.e., Supportive Relationships, Rules, Precorrection, Opportunities to Respond, Praise). 
Second, it includes items that address the relational factors of treatment integrity: Disruptions and 
Student Responsiveness; Figure 1). Finally, it includes items not included as part of BEST in CLASS 
but that have some evidence of importance for the outcomes and developmental trajectories of 
students. Specifically, McLeod, Kunemund, Nemer, and Lyon (2019) used a practice elements 
distillation approach to identify common practice elements from evidence-based programs and 
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interventions, and these common practice elements comprise the TIES measure. To illustrate, 
additional items that are not part of BEST in CLASS include self-management, peer tutoring, and 
error correction. Importantly, this adaptation to the TIES will allow for the assessment of an 
additional integrity dimension, treatment differentiation (Figure 1). 

In addition to the initial psychometric study of the BiCACS, both measures have been used 
reliably and have shown to be sensitive to change. To illustrate, using Cicchetti’s (1994) criteria, 
Adherence item intraclass correlation coefficients (ICCs) on the BiCACS in the BEST in CLASS-PK 
trial (Sutherland et al., 2018) ranged from “fair” to “‘excellent” (ICCs ranged from .40 to .83 
[M = .71, SD = .13]), while for the Competence subscale item ICCs ranged from “‘poor’”” to “‘good” 
(ICCs ranged from .12 to .62 [M = .37, SD = .15]). Effect sizes at posttest for Adherence (ES = 1.30) 
and Competence (ES = 2.88) were large. In the BEST in CLASS-E pilot study (Sutherland, Conroy, 
McLeod, Granger et al., 2019), the mean ICC for the Adherence scale of the TIES was .82 (SD = 0.10; 
range .68 to .92), with all items reflecting “good” to “excellent” agreement. The mean ICC for the 
Competence scale was .61 (SD = .11; range .52 to .77), with items representing “fair” to “excellent” 
agreement. Effect sizes at posttest were moderate to large (.67 and 1.70) for Adherence and 
Competence prescribed items, respectively. 

The development of the BiCACS and the TIES were important as we conceptualized how to assess 
teacher delivery of the BEST in CLASS practices in early childhood and elementary school class- 
rooms. In the development of these instruments, we considered a theoretical model of treatment 
implementation (see Figure 1), which proposes pre-treatment conditions, such as school character- 
istics, child characteristics, and teacher characteristics, influence the extent to which treatment 
implementation occurs. In turn, treatment implementation is thought to link with post treatment 
outcomes, such as behavioral, emotional, social, and academic outcomes for students. This model 
has guided recent work investigating the barriers and facilitators of BEST in CLASS implementation. 


29 «ce 


Barriers and facilitators of implementation of BEST in CLASS 


Domitrovich et al. (2008) proposed a multi-level framework of factors that may influence the quality 
of program implementation in schools. This model is consistent with a socio-ecological framework 
(Bronfenbrenner, 1977) and takes into consideration the influences of macro-level, school-level, and 
individual-level factors. Macro-level factors are conceptualized as the broadest set of factors that 
have the potential to influence the quality of implementation within schools (e.g., government 
policies, community supports, funding). The second, school-level, recognizes the school is an 
organization in which teachers, children, and staff are embedded in a shared environment with 
the potential to influence program implementation. Factors at this level can include school policies, 
school climate, and school resources. The final level, individual-level, represents teacher or child level 
factors that can promote or undermine the quality of intervention implementation in schools. 
Examples of individual level factors that may influence implementation include teachers’ character- 
istics (e.g., self-efficacy, level of education), psychological functioning (e.g., stress, depression), and 
perception and attitudes toward the intervention (e.g., intervention acceptability). 

In Domitrovich et al.’s (2008) model, factors at each level are conceptualized as interdependent 
with the potential to influence the quality with which interventions are implemented, although 
proximal factors are thought to be more influential than distal factors. Considering the dynamic 
interplay of these factors highlights the complexity and difficulty of implementing evidenced-based 
practices in authentic settings such as classrooms. Classrooms are complex settings in which 
interventions must adapt to fit the multidimensional needs of both teachers and children. 
Therefore, in order to create and adapt future teacher-delivered intervention programs that are 
feasible and sustainable, it is critically important to gain a better understanding of the barriers and 
facilitators of implementation in classroom settings. In the next section we describe the barriers and 
facilitators of implementation of BEST in CLASS-PK and BEST in CLASS-E. 
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BEST in CLASS-PK 

In light of the complexities inherent in school based programs and teacher delivery of interventions, 
Sutherland, Conroy, McLeod et al. (2018) explored classroom and teacher factors that influence the 
treatment integrity of BEST in CLASS-PK. Drawing on Domitrovich et al.’s (2008) conceptual 
model, which suggests proximal factors may influence implementation, these researchers examined 
the extent to which observations of classroom quality (i.e., CLASS), and levels of child problem 
behavior influenced teacher delivery of BEST in CLASS-PK. Additionally, they examined the extent 
to which teachers’ self-efficacy and educational background influenced their implementation of 
BEST in CLASS-PK. 

The classroom, teacher, and treatment integrity data came from the treatment group (n = 92 
teachers; n = 231 children) in the efficacy trial of BEST in CLASS-PK (Sutherland, Conroy, Algina 
et al., 2018). Results revealed both adherence and competence increased across the 14-week coaching 
process, as expected. Teachers with higher levels of education (i.e., categorized as high school 
diploma or associates degree, bachelor’s degree, or master’s degree or higher) delivered the program 
with more adherence and competence initially, suggesting teachers with more education and training 
are more capable of implementing a Tier-2 intervention such as BEST in CLASS than teachers with 
less education and training. However, teachers with lower levels of initial competence exhibited 
more growth in competence over time than teachers with higher levels of initial competence. 
Teachers with higher initial scores on the Classroom Organization subscale of the CLASS exhibited 
lower growth in adherence across time, while teachers with higher initial scores on the Emotional 
Support subscale implemented the program with greater competence and had higher growth in both 
adherence and competence across time. Contrary to hypotheses teacher self-efficacy did not predict 
adherence, and teachers who reported higher initial levels of Student Engagement self-efficacy 
exhibited lower growth in competence over time. Finally, child problem behavior was not associated 
with teacher implementation of BEST in CLASS practices. 

These findings highlight the importance of considering connections between classroom, teacher, 
and child-level factors and teacher implementation of prevention programs. In regards to classroom 
factors, Sutherland, Conroy, McLeod et al. (2018) theorize it is plausible teachers with higher ratings 
of Classroom Organization were already performing many of the BEST in CLASS practices related to 
Classroom Organization (e.g., Rules, Praise, OTR) and had less need (and capacity) for growth in 
these areas. However, teachers with higher initial ratings of Emotional Support delivered the 
program with greater competence initially and had greater growth in both adherence and compe- 
tence over time, suggesting teachers with higher initial levels of Emotional Support may have 
attempted to improve the quality of delivery of BEST in CLASS practices because they anticipated 
the practices would exert a positive impact on children’s social and emotional development (i.e., they 
were more attuned to the needs of their children). 

Turning to teacher-level factors, results suggest more education may help teachers transfer 
information more rapidly from the initial training to their classroom and teachers with lower levels 
of professional preparation may benefit the most from professional development, such as the use of 
practice-based coaching supports. Indeed, teachers with lower levels of initial competence had 
greater growth in competence of delivery than did teachers with higher levels of initial competence. 
However, teachers with higher levels of self-efficacy did not exhibit higher initial implementation of 
BEST in CLASS. In fact, teachers with higher initial levels of Student Engagement self-efficacy 
exhibited lower growth in competence; it is possible teachers with higher Student Engagement self- 
efficacy were already aware of how their instructional behavior was related to engaging children in 
learning, and had less room for growth in terms of implementing the BEST in CLASS practices. 
Taken together, interventions such as BEST in CLASS may yield higher levels of implementation 
quality and subsequently better outcomes when working with teachers who have less education or 
lower initial quality of instructional practice delivery. However, more research is needed to better 
understand the relations between teacher self-efficacy and implementation. 
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Finally, at the child-level, child problem behavior was not associated with teacher implementation of 
BEST in CLASS. This may be due to a lack of variability in the problem behavior of the children; all 
children were screened into the study by meeting a threshold of problem behavior indicative of risk for 
EBDs. Thus, teachers in the study were focusing their practices on children with similar levels of problem 
behavior. Regardless, this is surprising given other interventions have demonstrated program effects 
were only present for children with high levels of problem behavior; for instance, significant reductions 
in aggression were found only for the high-risk, aggressive children (e.g., Conduct Problems Prevention 
Research Group, 2002). Similarly, other interventions with a focus on academic skills indicate the most 
significant effects of high quality care are among children with fewest socioeconomic resources who often 
enter preschool cognitively behind their peers (Peisner-Feinberg & Burchinol, 1997). This work suggests 
intervention efforts may be most helpful to children who begin at low levels of psychosocial functioning 
or high levels of problem behavior in the context of Tier-1 programs. However, less is known about the 
extent to which entry level skills or behaviors influence outcomes in the context of Tier-2 programs (e.g., 
Domitrovich et al., 2008; Pas, Cash, O’Brennan, Debnam, & Bradshaw, 2015). Future work is needed to 
understand differences in child-level barriers and facilitators to implementation across Tier-1 and Tier-2 
programs. 


BEST in CLASS-E 

BEST in CLASS-E was adapted from the framework of BEST in CLASS-PK through an iterative 
process (see Sutherland, Conroy, McLeod, Granger et al., 2019). It was adapted to enhance the 
feasibility of implementation and sustainability in early elementary classroom settings (kindergarten- 
34 grade). Through this iterative process we uncovered facilitators and barriers to implementation 
and made several adaptations to the BEST in CLASS-PK framework to be responsive to the needs of 
elementary school teachers and students. Next we discuss developmental and implementation 
considerations and data from a feasibility study that was used to inform the final implementation 
model (see Sutherland et al., 2019 for a full review of model adaptation). 

The first implementation barrier we encountered in the initial adaptation was aligning BEST in 
CLASS-E with developmentally appropriate practices for elementary school teachers and students. 
Salient differences exist between early childhood and elementary school contexts, which required 
a shift from practices appropriate for early childhood teachers and an early childhood context to 
practices that fit elementary school teachers and classrooms. For instance, preschool-age curricula 
focus upon implementation of developmentally appropriate practices (Copple & Bredekamp, 2009; 
Wolery & Hemmeter, 2011) and elementary school maintains a stronger focus on academic 
achievement (Sutherland, Lewis-Palmer, Stichter, & Morgan, 2008). Therefore, a particular focus 
of BEST in CLASS-E intervention development, and a significant change from the focus of BEST in 
CLASS-PK, was the identification of evidence-based practice elements (i.e., Supportive Relationships, 
Rules, Precorrection, Opportunities to Respond, Behavior-Specific Praise, and Home-School 
Partnership; see Sutherland, Conroy, McLeod, Kunemund et al., 2019) that are associated not only 
with student social-emotional learning and growth but also with student academic growth. 

In addition to the developmental needs of early elementary students, we also considered barriers 
to implementation in regards to differences between early childhood and elementary school contexts. 
Early childhood classrooms are often comprised of a range of teacher-led learning activities and 
child-directed learning activities throughout the day (e.g., center-time, play, art, music, small group 
activities), while elementary school classrooms are more structured, giving students less time for 
child-directed learning activities and more time for teacher-led academic content. Additionally, 
elementary school teachers tend to have more formal professional preparation and training, teaching 
credentials, and certifications than teachers in early childhood settings (Pianta & Rimm-Kaufman, 
2006). Early childhood and elementary school teachers also may have different behavioral expecta- 
tions for their students based in large part on the different developmental and ecological contexts of 
early childhood classrooms versus elementary school classrooms and their training backgrounds 
(Pianta & Rimm-Kaufman, 2006). 
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To test these adaptations to BEST in CLASS-E we conducted a feasibility study, including eight 
teachers who were trained and coached on BEST in CLASS-E with 14 focal students (one to two 
students per classroom). Data collection included (a) direct observations of teacher and student 
behavior, including teacher implementation integrity, (b) teacher reports of student behavior and 
teacher self-efficacy, (c) teacher and coach ratings of effectiveness and feasibility of the practice 
elements, (d) teacher and family structured interviews, and (d) a coach focus group. Results indicate 
teachers’ adherence and competence to the BEST in CLASS-E practice elements improved from pretest 
to posttest, serving as an initial indicator that the adaptations to BEST in CLASS-E were responsive to 
the barriers raised in the iterative design process. Teachers also reported small mean increases on the 
three subscales of the Teacher Sense of Self-Efficacy Scale (Tschannen-Moran & Hoy, 2001) from 
pretest to posttest. Additionally, positive changes were noted across subscales of the Child Behavior 
Checklist (CBCL; Achenbach & Rescorla, 2001) and Social Skills Improvement System (SSIS; Gresham 
& Elliott, 2008) for focal students. While these data suggest training and coaching in the BEST in 
CLASS-E practice elements led to increased and higher quality delivery and some promise on student 
outcomes, qualitative data also informed adaptations to finalize the BEST in CLASS-E model. 

Following the feasibility study, we conducted interviews with the eight teachers involved in the 
feasibility test, three coaches and 11 of the 14 families of focal students identified as at risk for 
EBD. Family interviews revealed some families felt less satisfied than teachers with the quality of 
home-school communication and there was a general lack of trust between families and teachers 
(Miller et al., 2017; Sutherland, Conroy, McLeod, Granger et al., 2019). At the same time, some 
families highlighted supports for home-school partnerships that included culturally sensitive 
collaboration with teachers that contributed to positive communication, and several families 
noted improved communication with their child’s teacher. Teacher interviews mirrored some of 
the family concerns including a lack of trust between teachers and families, as well as highlighting 
the critical role played by BEST in CLASS coaches in facilitating meetings and ongoing colla- 
borative relationships between families and teachers. Last, both coach and teacher interviews 
highlighted challenges with scheduling coaching meetings, indicating finding time during the 
school day for these meetings was difficult. These findings highlighted the importance of coaching 
as an implementation support for teachers to deliver the practice elements with focal students 
with adherence and competence, while acknowledging implementation barriers that may inhibit 
this important component. 

Relatedly, the feasibility study also signaled that program implementation and success was influenced 
by principal support. In schools where the principal encouraged a partnership with BEST in CLASS 
teacher participation rates were higher and teachers were more amendable to begin working with their 
BEST in CLASS coach. Additionally, principal support helped to protect teacher’s time for coaching 
meetings during the school day and in turn supporting teachers’ participation in BEST in CLASS, which 
is important considering that this was a barrier identified by teachers. 

Importantly, social acceptability findings indicate teachers felt comfortable with the intervention and 
training they received through BEST in CLASS and reported the intervention was highly useful for their 
classrooms (Conroy et al., 2014). To illustrate, in BEST in CLASS-PK teachers reported the intervention 
to be only minimally difficult to implement and minimally disruptive to their classroom. Teachers also 
reported the intervention required minimal to moderate amounts of time. Finally, they indicated that the 
intervention was highly useful in improving their classroom atmosphere and decreasing classroom 
problem behavior and that they were highly likely to continue using the BEST in CLASS intervention. 


Discussion 


Overall, recent findings from both BEST in CLASS-PK and BEST in CLASS-E indicate a need for 
focused efforts on examining the individual child- and teacher-characteristics, as well as classroom 
contexts, that may be leveraged to increase the likelihood of effective and successful program 
implementation, with a particular focus on using these factors to counterbalance a tendency toward 
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poor implementation of programs in authentic school settings. Findings inform not only the future 
development, implementation, and testing of BEST in CLASS but may also be generalized to other 
Tier-2 interventions with similar groups of children with and at risk for EBD. 


Importance of treatment integrity 


First, findings from previous studies solidify the importance of measuring and assessing treatment 
integrity as part of program evaluation. Our knowledge about BEST in CLASS-PK and BEST in CLASS- 
E is greatly enhanced by intentionally including continuous measures of multiple dimensions of 
treatment integrity and assessing these dimensions at several time points throughout the intervention 
in both treatment and control conditions. By going beyond a checklist style of measuring treatment 
integrity, BEST in CLASS avoids truncating the variance needed to assess program implementation 
(Sutherland et al., 2013). Moreover, assessing treatment integrity opens the door for research questions 
about factors that may influence implementation such as why variation in implementation exists, why 
variation in program outcomes exists, and how to best support implementation and outcomes across 
a range of settings and participants. Finally, future work may look to examine thresholds at which 
implementation quality begins to influence program outcomes, and supports that promote or hinder 
teachers’ ability to deliver programs at each threshold. Treatment integrity measures that allow for 
variability across dimensions of integrity, such as the BiCACS and TIES, allow for such research. 


BEST in CLASS implementation and individual level factors 


Second, consistent with Domitrovich et al.’s (2008) model, this review emphasizes the importance of 
considering the connection between individual-level factors and quality of implementation. Across 
both BEST in CLASS-PK and BEST in CLASS-E teacher education and training were related to 
program implementation. In the PK trial, teacher education level influenced initial adherence and 
competence to the practices, while in the development of the elementary program significant 
adaptations to the model were needed based on education and training differences between early 
childhood and elementary school teachers. These findings are consistent with prior work suggests 
a relation between professional characteristics of service providers and program implementation. For 
example, a higher level of education in mental health clinicians was associated with more positive 
attitudes toward evidence-based practices (Aarons, 2004). However, other extant literature suggests 
mixed findings, hinting that years of education or experience may not be related to program fidelity 
(Ringwalt et al., 2003, 2002). Because teachers can vary widely in their education, experience, and 
expertise, it is important to continue to examine the influence of teacher education on program 
implementation. It will also be important for future work to examine growth in adherence and 
competence over time and how individual teacher level factors such as initial scores on adherence 
and competence influence growth trajectories. 


BEST in CLASS implementation and classroom contextual factors 


In addition to teacher-level factors, classroom quality also appears to influence treatment integrity of 
BEST in CLASS. In the PK trial, higher initial classroom organization predicted lower growth in 
adherence across time and higher initial emotional support scores predicted greater growth in both 
adherence and competence across time. Findings underline the importance of considering classroom 
context as a facilitator of treatment implementation. That is, we may be better positioned to 
implement programs such as BEST in CLASS by screening teachers for indicators of classroom 
quality (e.g., levels of classroom organization and emotional support) prior to program implementa- 
tion, and then tailor coaching supports for teachers based on initial scores of classroom quality. In 
our future work we intend to examine the extent to which classroom level risk may influence teacher 
delivery of the program; prior work suggests classroom contexts (e.g., poverty-level of the 
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surrounding community, teacher turnover) may influence implementation quality (Durlak & DuPre, 
2008). Preliminary findings from BEST in CLASS-E document our student population faced a high 
level of cumulative risk factors (e.g., toxic stress; learning problems; see Sutherland, Conroy, 
McLeod, Granger et al., 2019). In future studies we plan to examine how classroom level risk 
composition may moderate teacher delivery of the BEST in CLASS practices and associated student 
outcomes. 


BEST in CLASS implementation and school level factors 


Finally, across both BEST in CLASS-PK and BEST in CLASS-E program implementation and success 
was influenced by school and program level factors such as principal support. To illustrate, schools in 
which principals were supportive and encouraged teacher participation and buy-in to program efforts 
were more likely to be amenable to scheduling coaching meetings and supporting training efforts. This 
is consistent with prior work supporting a significant main effect for principal support and a significant 
interaction between principal support and the fidelity of teacher’s implementation on student outcomes 
in a school based mental health program; when both of these factors were high, students improved 
significantly on all outcomes (Kam, Greenberg, & Walls, 2003). Additionally, supportive principals with 
a general positive feeling toward the program help the program run smoothly in the school, rather than 
acting indifferently or even creating obstacles to implementation, resulting in higher quality imple- 
mentation (Berends et al. 2002; Farrell, Meyer, & White, 2001; Kam et al., 2003; Payne, 2009; Payne, 
Gottfredson, & Gottfredson, 2006). In future work, researchers must recognize the importance of 
principal and teacher buy in for successful program outcomes and sustainability. 


Conclusion 


Sustaining high-quality levels of BEST in CLASS treatment integrity by teachers continues to be 
a challenge. Factors such as individual child- and teacher-characteristics, as well as classroom 
contexts are linked to both increases and decreases in the likelihood of effective and successful 
implementation. Continued research is needed to more fully understand connections between child, 
teacher, and classroom factors, treatment integrity, and implementation quality. The experiences 
implementing BEST in CLASS-PK and BEST in CLASS-E serve as a springboard from which future 
intervention work can consider barriers and facilitators to delivery and outcomes when designing 
effective programs. As such, the current paper is one more step forward in the process of better 
understanding these relations, and one that will hopefully lead to more efficient and sustainable 
models of program delivery in early childhood and elementary school settings. 
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